feat(font): CPU-side font/glyph rendering pipeline + workspace module split#54
feat(font): CPU-side font/glyph rendering pipeline + workspace module split#54tonyfettes wants to merge 12 commits into
Conversation
`Stream::emit_sgr` allocated a fresh `SgrParser` for every CSI `m` via `from_params`, then threw it away after iterating the attributes. SGR is one of the hottest CSI sequences, so that is one heap object (plus an `ArrayView` incref) per sequence, dropped immediately after. Keep a single `SgrParser` on the `Stream` and re-point it at each new parameter view with a new `set_view` (a non-copying counterpart to `from_params`: assigns the borrowed view + resets the cursor). Same synchronous-dispatch lifetime contract as before; the parser just outlives one sequence now, like the parser's own reused accumulators. No behavior change. Net public API: +`SgrParser::set_view` (the new `Stream` field is private; Stream is abstract). Benchmark (same-session, multi-run mean): sgr_storm ~6.25ms -> ~5.5ms when stacked on the dispatch fast path Full suite: all tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring feihaoxiang/raster in-tree as `tonyfettes/ghostty/font/raster`, the glyph-outline fill stage of the font subsystem — the pure-MoonBit stand-in for what Ghostty delegates to FreeType in font/face/freetype.zig. Analytic signed-area coverage (the ftgrays / stb_truetype v2 cell-accumulation method): each edge walked once through the pixel grid depositing signed cover/area per cell, resolved by a left-to-right prefix sweep. Self-validating with no reference images — area conservation vs shoelace, a point-sampled supersampling oracle, fill-rule discrimination, and an ASCII snapshot. Depends only on moonbitlang/core/math, so it stays target-agnostic (wasm-gc + native) and adds no dependency footprint to the project. The harfbuzz->raster FreeType differential gate stays in the external feihaoxiang/glyph_pipeline harness for now; it folds in when the font/face/ shaping layer lands. 12 tests pass; ramp lookup switched to FixedArray for prefer_readonly_array. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MoonBit port of Ghostty's font/Atlas.zig: a square grayscale/BGR/BGRA texture
atlas with skyline ("shelf") rectangle bin-packing (Jukka Jylänki / freetype-gl)
and a 1px sampling border. API mirrors the original — new/clear/reserve/set/
set_from_larger/grow — with reserve raising AtlasFull when no spot fits.
Opaque Atlas; the GPU uploader reads state through accessors (data/size/format/
modified/resized) rather than public mutable fields. The modified/resized
counters let an uploader tell an in-place update from a reallocation.
Tests port Ghostty's suite verbatim (same sizes and byte offsets: exact fit,
doesn't-fit, fit-multiple, write, write-from-larger, grow, BGR) and add packing
invariants with no reference image — across thousands of reservations, regions
stay inside the border and never overlap.
Pure MoonBit, depends only on core; stays target-agnostic. Glyph metadata and
the glyph cache land next, when the rasterize->reserve->set producer exists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the texture atlas into a dedicated `font/atlas` package (tonyfettes/ghostty/font/atlas) and rename Atlas.mbt -> atlas.mbt to follow MoonBit's lowercase file-naming convention (the capitalized name was a holdover from the Zig source). No code changes; 9 tests still pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire raster and atlas together. `render_glyph` takes a @raster.Coverage, crops it to the inked bounding box, reserves an atlas region of exactly that size, writes the cropped grayscale bitmap, and returns Glyph metadata (tight size, bbox offset, atlas position). A blank coverage (e.g. a space) yields a zero-sized glyph and consumes no atlas space. GlyphCache is the SharedGrid analog: an atlas plus a memo table keyed by GlyphKey (font, glyph id, px). `render` rasterizes a glyph into the atlas once and returns the cached entry on every later lookup; the coverage is only consulted on a miss. This layer is font-agnostic — it never touches a font, so it stays pure and target-agnostic; the harfbuzz outline source plugs in later. Adds Coverage::width/height accessors to raster so an out-of-package consumer can crop. Tests drive the chain with synthetic shapes (no font): tight-crop correctness, blank glyph, anti-aliased byte quantization, cache hit/miss + atlas-untouched on hit, and disjoint placement of distinct glyphs. 5 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the font face — the real outline source — wiring harfbuzz (a pure-MoonBit HarfBuzz port) into the stack. `Face::load` takes in-memory font bytes; `glyph_index` resolves a codepoint through the cmap; `rasterize` walks the glyph outline via harfbuzz `draw`, flattens the Béziers through raster's PathBuilder under a font-unit→pixel transform (scale + Y-flip + baseline), and fills it to a coverage buffer. This is the MoonBit analog of font/face/freetype.zig, except the rasterizer is our own. harfbuzz is added as a registry dependency (moonbit-community/harfbuzz@0.1.0). It is pure MoonBit, so the project stays target-agnostic (wasm-gc + native) — no filesystem, no native, no C FFI. Font bytes are embedded: a fully-instantiated, 6-glyph subset of Inconsolata (1068 bytes) checked in as a MoonBit Bytes literal, mirroring harfbuzz.mbt's own *_gen.mbt fixtures, with the generator script under tools/. Tests drive the whole font stack end to end with the embedded font, no disk: cmap hit/miss, real 'A' rendered through harfbuzz → raster → GlyphCache with an inked atlas pixel, disjoint multi-glyph packing + free cache hits, and an ASCII snapshot of the rendered 'A'. The external glyph-pipeline keeps the FreeType differential gate. 5 tests; 31 across the font tree. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lls)
The first junction of terminal and font. `build` walks a terminal RenderState
snapshot row by row and produces the flat per-cell instance data a GPU draws
with one instanced call — the MoonBit analog of renderer/cell.zig's Contents:
* CellText: one textured-glyph quad instance (grid position, atlas position
+ size, bearings, foreground color, atlas kind), sparse — one per inked cell.
* Rgba / CellBg: a dense background-color grid, one per cell.
The screen→data mapping lives here so it is testable offline; it touches no GPU.
Glyph lookup is a caller-supplied resolver `(UInt) -> Glyph? raise`, keeping the
builder decoupled from the font internals — the real resolver wires face +
GlyphCache, a test can pass any closure. Blanks and spaces contribute background
only. Contents is opaque, read via accessors as the uploader will.
Also promote the embedded subset font from a test-scoped binding in font/face to
public data (font/face/embedded_font.mbt, the font/embedded.zig analog) so the
renderer tests can render real glyphs.
Tests drive the whole stack end to end with a real StreamTerminal: type text,
snapshot a RenderState, build instances — asserting one glyph per inked cell,
spaces skipped, dense background, default colors, distinct atlas placement, and
the None-resolver path. 4 tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dissolve the single tonyfettes/ghostty module into three workspace
members so native-only dependencies stay out of the terminal core:
tonyfettes/ghostty-terminal (terminal/) — no native deps, builds on
wasm/wasm-gc/js/native
tonyfettes/ghostty-font (font/) — owns the harfbuzz dep
tonyfettes/ghostty-renderer (renderer/) — deps terminal + font
The harfbuzz import moves from the root moon.mod into font/moon.mod, so
the terminal parser/grid no longer share a dependency set with the font
pipeline. Root module is dissolved; the repo top is now a pure moon.work
workspace. Import paths rewritten (ghostty/terminal -> ghostty-terminal,
ghostty/font -> ghostty-font); the renamed terminal root package gets an
explicit @Terminal alias at its 5 call sites. examples/bench depend on
ghostty-terminal@0.3.0. .mbti regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e97b5a97da
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
| let gw = max_x - min_x + 1 | ||
| let gh = max_y - min_y + 1 | ||
| let bytes = Array::make(gw * gh, b'\x00') |
There was a problem hiding this comment.
When render_glyph is used with a Bgr or Bgra atlas, this buffer is only one byte per pixel, but Atlas::set copies region.width * region.height * atlas.format().depth() bytes. Since both non-grayscale formats are public and GlyphCache::new accepts any Format, a non-empty glyph in such a cache will read past the source buffer (or corrupt the atlas data) instead of writing valid multi-byte pixels; either restrict glyph rendering to grayscale atlases or expand the source buffer by the atlas depth.
Useful? React with 👍 / 👎.
| /// spaces contribute background only, exactly as the GPU renderer expects. | ||
| pub fn build( | ||
| state : @terminal.RenderState, | ||
| glyph_of : (UInt) -> @glyph.Glyph? raise, |
There was a problem hiding this comment.
The terminal snapshot can store a base codepoint plus combining/emoji modifiers in RenderRowCells::cell_grapheme_codepoints(), but this resolver API only receives the base UInt. For inputs such as e + U+0301 or an emoji skin-tone modifier, build can only render the uncombined base glyph (and a space with combining marks is later treated as a blank), so the renderer loses grapheme content that the terminal model already preserves; pass the grapheme extras through the glyph resolver/shape key.
Useful? React with 👍 / 👎.
| pub fn Atlas::data(self : Atlas) -> Array[Byte] { | ||
| self.data |
There was a problem hiding this comment.
This exposes the atlas's live mutable Array[Byte] even though the API documents it as read-only. Any uploader or external consumer can mutate texture bytes directly without going through set/clear, so modified() is not bumped and dirty tracking can skip a required upload; return a ReadOnlyArray[Byte] or a copy from the public surface.
Useful? React with 👍 / 👎.
build() resolved each cell's fg/bg straight from the style and never consulted cell_style().inverse(), so ESC[7m text drew with normal colors. Resolve fg/bg to concrete colors (defaults included) first, then swap on inverse — matching Ghostty's renderer/generic.zig, which inverts at the cell-build layer. Adds a regression test (inverse 'a' -> black glyph on a white cell). Combining marks remain dropped (only the base codepoint is resolved); documented as a TODO since full grapheme rendering needs run-based HarfBuzz shaping, not a per-cell single-glyph lookup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c8a02cbf86
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // from cells.cell_grapheme_codepoints() are dropped, so clusters render as | ||
| // the bare base glyph. Rendering the marks needs run-based shaping (one | ||
| // cell -> several positioned glyphs), not this per-cell single-glyph lookup. | ||
| if glyph_of(cp) is Some(g) && g.width > 0 && g.height > 0 { |
There was a problem hiding this comment.
When a cell has SGR 8, CellStyle::invisible() is set and the existing formatter hides/re-emits it (terminal/formatter.mbt handles style.invisible()), but this renderer only reads the style for inverse and still reaches glyph_of/text.push for any non-space codepoint. In GPU paths built from build, concealed text such as SGR 8 hidden/password fields will be rendered visibly; skip foreground emission or make it transparent when cells.cell_style().invisible() is true.
Useful? React with 👍 / 👎.
render_glyph builds a 1-byte-per-pixel grayscale bitmap from scalar
coverage, but atlas.set copies width*height*depth bytes. A GlyphCache
created with a Bgr/Bgra format would therefore read past the source
buffer on the first inked glyph (codex P2 at glyph.mbt:90).
Guard the format and raise a typed WrongAtlas instead, mirroring
Ghostty's face.renderGlyph, which returns error.WrongAtlas when a
grayscale bitmap meets a non-grayscale atlas (color glyphs use a
separate bitmap path). The error lives in font/glyph, not font/atlas:
Ghostty's Atlas error set is error{AtlasFull} only, and a format
mismatch is a property of what is being rendered, not of reserve — so
reserve callers are not forced to handle it. render_glyph and
GlyphCache::render now raise the general error (AtlasFull + WrongAtlas).
Also document Atlas::data as deliberately returning the live array
(not a copy): the GPU uploader streams it straight to the texture,
exactly as Ghostty reads atlas.data in replaceRegion(.., atlas.data)
(codex P3 — a read-only wrapper would diverge from upstream and add a
copy on the upload path).
The grapheme-cluster gap (codex P2 at build.mbt:23) stays a documented
TODO: rendering combining marks needs run-based HarfBuzz shaping, as in
Ghostty's font/shaper, which is out of scope for this CPU-side builder.
Adds a regression test that a Bgra-backed render_glyph raises WrongAtlas.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff3de0682c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // from cells.cell_grapheme_codepoints() are dropped, so clusters render as | ||
| // the bare base glyph. Rendering the marks needs run-based shaping (one | ||
| // cell -> several positioned glyphs), not this per-cell single-glyph lookup. | ||
| if glyph_of(cp) is Some(g) && g.width > 0 && g.height > 0 { |
There was a problem hiding this comment.
When a cell is styled with SGR 1/3 (bold/italic), the terminal snapshot still exposes those flags through cells.cell_style() and the formatter preserves them, but this glyph lookup only passes the base codepoint. A GPU path built from build therefore has no way to select a bold/italic face or cache key and will render styled text with the same regular glyph; pass the relevant CellStyle/font style into the resolver or include it in the emitted instance data.
Useful? React with 👍 / 👎.
Invisible cells drew their glyph visibly. Match xterm and Ghostty's renderer (generic.zig: `if (style.flags.invisible) continue;` after the bg write): draw the background, then emit no foreground glyph, so concealed text such as password fields never reaches the GPU glyph path. Also fold the deferred style-aware work (grapheme clusters, bold/italic face selection) into one docstring note — both need the not-yet-wired font/shaper, since the bare-codepoint resolver can't select a face. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 682dea17f4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| pub fn GlyphCache::atlas(self : GlyphCache) -> @atlas.Atlas { | ||
| self.atlas |
There was a problem hiding this comment.
When consumers use this accessor for the documented GPU upload path, they receive the same mutable Atlas that the cache's memo table depends on. If that consumer calls public mutators such as cache.atlas().clear() or writes scratch regions through set, GlyphCache::render will still take the cache-hit branch and return old glyph positions even though those pixels were erased or overwritten; expose a read-only upload view or cache-owned mutation methods that keep glyphs in sync.
Useful? React with 👍 / 👎.
1 participant
What
Builds the CPU-side font/glyph rendering pipeline for the terminal (the MoonBit port of Ghostty's
font/*+renderer/cell.zig), then restructures the repo into three independent workspace modules so the renderer's native-leaning dependencies never reach the terminal core.The pipeline (bottom-up)
GPU upload itself is intentionally not included — the pipeline only produces arrays, so it stays offline-testable.
Workspace module split
The single
tonyfettes/ghosttymodule is dissolved into three workspace members:tonyfettes/ghostty-terminalterminal/tonyfettes/ghostty-fontfont/tonyfettes/ghostty-rendererrenderer/The harfbuzz dependency moves out of the root manifest into
font/moon.mod, so the terminal parser/grid no longer share a dependency set with the font pipeline — keeping the terminal core lean and multi-backend as native font/GPU bindings are introduced later. Repo top is now a puremoon.workworkspace.Verification
moon check— full workspace, cleanmoon -C terminal check --target all— terminal builds on wasm / wasm-gc / js / nativemoon fmt+moon inforun;.mbtiregenerated🤖 Generated with Claude Code